Reducing scale estimate errors in shelf images

ABSTRACT

Example image processing methods, apparatus/systems and articles of manufacture are disclosed herein. An example apparatus includes an image recognition application to identify matches between stored patterns and objects detected in a shelf image, where the shelf image has a shelf image scale estimate. The example apparatus further includes a scale corrector to calculate deviation values between sizes of (A) a first set of the objects detected in the shelf image and (B) a first set of the stored patterns matched with the first set of the objects and reduce an error of the shelf image scale estimate by calculating a scale correction value for the shelf image scale estimate based on the deviation values.

FIELD OF THE DISCLOSURE

This disclosure relates generally to image recognition in shelf imagesand, more particularly, to reducing scale estimate errors in shelfimages.

BACKGROUND

Image recognition applications or programs may be used to identifyconsumer packaged goods. For example, when performing an in-store shelfaudit, a market research firm may use an image recognition applicationto identify the products displayed on store shelves. Shelf auditing canbe used to report the items (e.g., goods) displayed in a store, wherethese items are displayed, how many times the times are displayed, etc.Image recognition applications or programs analyze a shelf image andattempt to match objects (e.g., shelf items) in the shelf image with oneor more reference images that are associated with particular products.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example image processing system capable offacilitating shelf auditing and constructed in accordance with theteachings of this disclosure.

FIG. 2 illustrates an example segmented image generated by an exampleimage recognition application of the example image processing system ofFIG. 1.

FIG. 3 is a flowchart representative of an example method, at leastpartially executable by machine readable instructions, which may beimplemented by the example image processing system of FIG. 1 to createan example pattern.

FIGS. 4A and 4B are flowcharts representative of an example method, atleast partially executable by machine readable instructions, which maybe implemented by the example image processing system of FIG. 1 toanalyze a shelf image.

FIG. 5 is a block diagram of an example processor system structured toexecute example machine readable instructions represented by FIGS. 3, 4Aand/or 4B to implement the example image processing system of FIG. 1.

DETAILED DESCRIPTION

Example image recognition methods, apparatus/systems and articles ofmanufacture that may be implemented to perform product auditing of ashelf image are disclosed herein. Example image processing systems andmethods disclosed herein achieve better recognition rates than knownimage recognition systems. As a result, an auditor may spend less timereviewing the results of an image recognition process and, thus, theexample systems and methods increase the efficiency of a shelf imageaudit process. Further, example image processing systems and methodsdisclosed herein more accurately determine the dimensions of theproducts in a shelf image, which can be used to enhance the imagerecognition process and generate more accurate reports regarding shelfspace.

In general, market research entities (MREs) and/or merchants (e.g.,wholesalers, club stores, retailers, e-commerce retailers) offerservices based on recognizing consumer packaged goods (e.g., products,items, objects, etc.) from images captured by image capturing devices(e.g., cameras, smartphones, tablets, video recorders, etc.). Forexample, an MRE may provide point-of-sale (e.g., shelf) auditingservices to marketers, manufacturers, merchants and/or advertisers tohelp determine if, for example, display locations of products and/ordisplay quality requirements (e.g., the goods are facing the correctorientation, the shelf is fully stocked, etc.) are satisfied. MRE and/ormerchants typically use an image recognition application (e.g., program,software, algorithm, etc.) that analyzes a point-of-sale (POS) image andattempts to detect and identify the products in the point-of-sale image.As used herein, the terms point-of-sale image, shelf image and storeimage are used interchangeably and mean any image having one or moreobjects (e.g., a consumer product), such as objects of interest to beaudited. A shelf image may contain only one shelf, a portion of a shelf,multiple shelves (e.g., in a column) and/or any other displayorientation where a product is presented to a consumer.

Prior image recognition applications analyze a shelf image and attemptto match objects (e.g., items, products, etc.) in the shelf image withone or more patterns that represent a known product. As used herein, apattern is an image representation of an object, such as a product, aportion of a product, a packaging of a product, a portion of a packagingof a product, etc. If a pattern is matched with an object in the shelfimage, the object can be associated with that product and theinformation can be used for, in some examples, auditing purposes.

Some prior image recognition applications fail to detect all of theproducts in the image and/or incorrectly identify products. This is due,in part, to circumstances where scaling information associated with thepatterns are not available or accurate. A scale of a pattern is a uniquerelationship between the dimensions of the pattern (in pixels (px)) andthe dimensions (in millimeters (mm)) of the object represented by thepattern. For example, a scale of a pattern may be 2 px/mm. Thus, everytwo pixels in the pattern represents 1 mm of the real dimension of theobject in the pattern. While the dimensions of the pattern (in pixels)are generally known, the dimensions of the object represented in thepattern are not always accurate. Thus, the scale of a pattern may beinaccurate. Further, the same is true with the scale of the shelf image.This inaccuracy in the true scale of the patterns and the shelf imagecauses problems during the image recognition process. In particular,once a shelf image scale estimate (in px/mm) is determined, the patternsare rescaled or resized based on a scaling ratio (e.g., a scalingfactor) to match the shelf image scale estimate. In general, imagerecognition processes are more accurate when the scale of a pattern isthe same as the scale of the shelf image, because the size of thepattern is presumably a 1:1 ratio of the size of the correspondingobject in the shelf image.

If the size of the pattern is different than the size of the object inthe shelf image, the image recognition process is less accurate, and maynot detect the object as corresponding to the correct pattern. Asmentioned above, the scales of the patterns and the scale of the shelfimage are often inaccurate and, thus, this leads to ineffective imagerecognition processes. As such, to ensure high accuracy output, anauditor (e.g., a user, an operator, a technician, etc.) manually reviewsthe results of the image recognition process to verify that all of theproducts in the image have been detected and/or that all of the productsare correctly identified. In some examples, identifying an object in ashelf image involves manually searching through a database containingthousands of products and then assigning an item number (SKU) and/orInternational Article Number (EAN) code to the area once a referenceproduct is located. This process of manually reviewing thousands ofproducts in a database and assigning SKU/EAN codes requires significanttime, processing resources, and/or experience on behalf of the auditor.

Example image processing systems and methods disclosed herein increasethe accuracy of the scale estimates of the patterns and the shelf imageand facilitate better recognition rates. Example image processingsystems and methods disclosed herein determine optimized scales forpatterns, which can be used to increase the accuracy during the imagerecognition process. As used herein, an optimized scale is a scale basedtwo or more scale estimates of a pattern. For example, some of thepatterns may have multiple scale estimates (e.g., from previous matchesin other shelf images). In some examples, the optimized scale is amedian of the scale estimates for the corresponding pattern. Thus, theoptimized scale is a more accurate value of the scale for the particularpattern. In some examples, the image processing system only calculatesoptimized scales for patterns having a threshold number of scaleestimates. If the threshold is two, for example, then optimized scalesare calculated for patterns having two or more scale estimates. If apattern has only one scale estimate, for example, the scale estimate maybe set to default or unknown, in which case the scale estimate for thatpattern is not relied on during the image recognition process (asdiscussed in further detail herein).

In some examples, prior to performing an image recognition process withan image recognition application, the example image processing systemrescales or resizes the patterns to the scale estimate of the shelfimage (e.g., based on a scale ratio). The image recognition applicationthen uses the rescaled/resized versions of the patterns in the imagerecognition process. The image recognition application attempts to matchthe patterns to one or more objects (e.g., products) in the shelf image.However, in some instances, the rescaled/resized versions of thepatterns are still different sizes (in pixels) than the correspondingobjects matched in the shelf image. In some examples, the example imageprocessing system determines an example scale correction value for theshelf image. In some examples, the scale correction value is based on astatistical measurement of the deviation values between the sizes of therescaled patterns having optimized scales and the corresponding objectsmatched in the shelf image. The example scale correction valuerepresents an overall deviation between the size of the patterns and thematched objects in the shelf image. The example scale correction valuecan be used to adjust or modify the shelf image scale estimate and/ordetermine another scale estimate for each the matched patterns, whichcan be added to, combined with, and/or otherwise considered with theexample scale estimates for the corresponding patterns and used insubsequent processes. After two or more iterations, the scale estimatesfor the patterns tend to converge to the true or accurate scales. Thus,with more accurate scale information for the shelf image and thepatterns, further image recognition processes become more accurate.

Example apparatus to reduce scale estimate errors are disclosed herein.Example apparatus include an image recognition application to identifymatches between stored patterns and objects detected in a shelf image,where the shelf image has a shelf image scale estimate. Some exampleapparatus further include a scale corrector to calculate deviationvalues between sizes of (A) a first set of the objects detected in theshelf image and (B) a first set of the stored patterns matched with thefirst set of the objects and reduce an error of the shelf image scaleestimate by calculating a scale correction value for the shelf imagescale estimate based on the deviation values.

In some examples, the scale corrector is to calculate the deviationvalues by determining a difference in pixel dimensions of a first scaledversion of a first pattern of the first set of the stored patterns and afirst object of the first set of the objects matched with the firstpattern. In some such examples, the example apparatus further includes arescaler to generate the first scaled version of the first pattern byrescaling the first pattern based on the shelf image scale estimate.

In some examples, the first set of the stored patterns have optimizedscales. In some examples, the apparatus further includes a pattern scaleoptimizer to determine the optimized scales for the first set of thestored patterns. In some examples, the optimized scales are a median ofestimated scales for the first set of the stored patterns. In someexamples, the stored patterns include a second set of the storedpatterns, where the first set of the stored patterns have a number ofscale estimates that satisfy a threshold, and the second set of thestored patterns have a number of scale estimates that do not satisfy thethreshold.

Example methods to reduce scale estimate errors are disclosed hereinthat include identifying, by executing an instruction with a processor,matches between stored patterns and objects detected in a shelf image,where the shelf image having an shelf image scale estimate. Examplemethods also include calculating, by executing an instruction with theprocessor, deviation values between sizes of (A) a first set of theobjects detected in the shelf image and (B) a first set of the storedpatterns matched with the first set of the objects, and reducing anerror of the shelf image scale estimate by calculating, by executing aninstruction with the processor, a scale correction value for the shelfimage scale estimate based on the deviation values.

In some examples, calculating the deviation values includes determininga difference in pixel dimensions of a first scaled version of a firstpattern of the first set of the stored patterns and a first object ofthe first set of the objects matched with the first pattern. Someexample methods further include generating the first scaled version ofthe first pattern by rescaling the first pattern based on the shelfimage scale estimate.

In some examples, the first set of the stored patterns have optimizedscales. Some such example methods further include determining theoptimized scales for the first set of the stored patterns. In someexamples, the optimized scales are a median of estimated scales for thefirst set of the stored patterns. In some examples, the stored patternsinclude a second set of the stored patterns, where the first set of thestored patterns have a number of scale estimates that satisfy athreshold, and the second set of the stored patterns have a number ofscale estimates that do not satisfy the threshold.

Tangible machine readable storage mediums are disclosed herein thatinclude instructions that, when executed, cause a machine to at leastidentify matches between stored patterns and objects detected in a shelfimage, where the shelf image having an shelf image scale estimate. Theinstructions further cause the machine to calculate deviation valuesbetween sizes of (A) a first set of the objects detected in the shelfimage and (B) a first set of the stored patterns matched with the firstset of the objects, and reduce an error of the shelf image scaleestimate by calculating a scale correction value for the shelf imagescale estimate based on the deviation values.

In some examples, the instructions, when executed, cause the machine tocalculate the deviation values by determining a difference in pixeldimensions of a first scaled version of a first pattern of the first setof the stored patterns and a first object of the first set of objectsmatched with the first pattern. In some examples, the instructions, whenexecuted, further cause the machine to generate the first scaled versionof the first pattern by rescaling the first pattern based on the shelfimage scale estimate.

In some examples, the first set of the stored patterns have optimizedscales. In some examples, the instructions, when executed, further causethe machine to determine the optimized scales for the first set of thestored patterns. In some examples, the stored patterns include a secondset of the stored patterns, where the first set of the stored patternshave a number of scale estimates that satisfy a threshold, and thesecond set of the stored patterns have a number of scale estimates thatdo not satisfy the threshold.

Turning now to the figures, FIG. 1 illustrates an example imageprocessing system 100 to perform image recognition in shelf images inaccordance with the teachings of this disclosure. The example imageprocessing system 100 may be implemented by an MRE. For example, theimage processing system 100 may be used in a workflow of an auditor toperform product auditing in a shelf image. In the illustrated example ofFIG. 1, an example image capturing device 102 (e.g., a camera, a videorecorder, a smartphone, a tablet, etc.) captures an example shelf image104 of an example shelf 106 having one or more products 108 a-n (e.g.,consumer packaged goods, items, objects, etc.). The example shelf 106may be associated with a merchant (e.g., a retailer, a wholesaler, aclub store, etc.). In other examples, the shelf 106 may be in anotherlocation, such as a consumer's home, in a vending machine, at a kioskand/or any other establishment (e.g., a museum, a gallery, etc.)containing items on display and that can be audited. In the illustratedexample of FIG. 1, the shelf image 104 is sent to the image processingsystem 100 through an example network 110 (e.g., the Internet, a localarea network, a wide area network, a cellular data network, etc.) via awired and/or wireless connection (e.g., a cable/DSL/satellite modem, acell tower, etc.).

In the illustrated example, the image processing system 100 includes anexample image recognition application 112 that analyzes the shelf image104 to identify the product(s) 108 a-n in the shelf image 104. The imagerecognition application 112 may be implemented by an application,software, engine, algorithm, hardware, etc. or any combination thereof.The image recognition application 112 attempts to recognize/identify andmatch the product(s) 108 a-n in the shelf image 104 with one or morepatterns 114 a-n (e.g., reference product images) stored in a patterndatabase 116. As mentioned above, the example patterns 114 a-n areimages of known products or portions of a product, including a package,a label, etc. Example patterns 114-n are associated with a product andmay include identifying information for the associated product. In someexamples, the patterns 114 a-n are provided by an external source (e.g.,a manufacturer, a reference image collection team, the Internet, etc.).The example image recognition application 112 compares the examplepatterns 114 a-n to objects in the shelf image 104 in an attempt tomatch the product(s) 108 a-n in the shelf image 104 with the productsassociated with the patterns 114 a-n. Example techniques that may beimplemented by the image recognition application 112 includeclassification techniques (parametric and non-parametric), clusteringtechniques (deep learning models, k-means clustering), multilinearsubspace learning techniques (principal component analysis), machinelearning techniques, etc. Additionally or alternatively, exampletechniques that may be implemented by the image recognition application112 include examples described in: U.S. patent application Ser. No.14/530,129, entitled “Context-Based Image Recognition For ConsumerMarket Research,” and filed Oct. 31, 2014, which corresponds to U.S.Patent Publication No. 2016/0125265, which published on May 5, 2016;U.S. Pat. No. 8,908,903, entitled “Image Recognition to Support ShelfAuditing for Consumer Research,” and granted on Dec. 9, 2014; and U.S.Pat. No. 8,917,902, entitled “Image Overlaying and Comparison forInventory Display Auditing,” and granted on Dec. 23, 2014, all of whichare incorporated herein by reference in their entireties.

In some examples, the results (e.g., matches) of the image recognitionprocess are viewed on an example user interface 118. In some examples,the image recognition application 112 creates or generates a segmentedimage (e.g., such as the segmented image 200 of FIG. 2, discussed infurther detail herein) indicating the results of the image recognitionprocess. Matches between the example patterns 114 a-n and the exampleproducts 108 a-n may be stored as match records in the pattern database116 and/or another database. Once the example image recognitionapplication 112 generates the segmented image, the segmented image maybe displayed on the user interface 118 for an auditor to review. Theexample user interface 18 may be implemented by any type of imagingdisplay using any type of display technology, such as a liquid crystaldisplay (LCD), a light emitting diode (LED) display, a plasma display, acathode ray tube (CRT) display, etc., one or more input devices (e.g., atouchscreen, touchpad, keypad, mouse, keyboard, etc.) and/or one or moreof the output devices 524 included in the example processor platform 500described in connection with FIG. 5.

In some examples, the results are used by the example image processingsystem 100 to generate shelf-audit reports. For example, as illustratedin FIG. 1, the results (e.g., the match records) of the imagerecognition process are used by an example report generator 120 togenerate an example report 122. In some examples, the report generator120 estimates the share of a shelf (or portion of a shelf) occupied by aparticular product or manufacturer of a product. The share of a shelf isa measure that represents the amount of space occupied in a shelf by aproduct or manufacturer. For example, based on the matches, the reportgenerator 120 may determine that Brand A occupies 20% of the shelf,Branch B occupies 10% of the shelf, Branch C occupies 10% of the shelf,and Brand D occupies 60% of the shelf. The example report 122 can beused by the manufacturers of Brands A, B, C and/or D to ensurecompliance with the contracts between the merchant and the manufacturers(e.g., based on an agreed quota for a particular shelf percentage).

However, as mentioned above, the example image recognition application112 may be less effective when the scales of the patterns 114 a-n arenot accurate. In some instances, the image recognition application 112may not detect an object in the shelf image 104 corresponding to one ofthe patterns 114 a-n and/or may falsely identify an object in the shelfimage 104 as corresponding to one of the patterns 114 a-n (e.g., a falsepositive). As such, auditors may spend numerous hours reviewing theresults of the image recognition process, attempt to manually identifythe product(s) shown in the shelf image 104, and/or utilize computingresources to correct errors. Labeling the products may involve manuallyand/or computationally intensive searches searching through one or moredatabases containing thousands of products, and then assigning an itemnumber, such as an SKU, and/or International Article Number (EAN) codeto the product depicted in a region of interest once a reference productis located. This process of manually reviewing thousands of products ina database and assigning SKU/EAN codes may require substantial time,computational resources, and/or experience on behalf of the auditor.

In some examples, patterns are created manually by an auditor. Forexample, an auditor may review a shelf image on the user interface 118.The auditor may highlight and/or otherwise select a region of interest(ROI) that defines a stock keeping unit (SKU) or a portion of a unit(e.g., a label) to save as a pattern. In the illustrated example of FIG.1, the image processing system 100 includes a pattern generator 124 thatdetects the highlighted ROI and saves an image associated therewith as apattern. After a pattern is created, identification information (e.g., abrand name) and/or a scale estimate may be associated with the pattern.In some examples, an auditor assigns identification information to thepattern by manually selecting the information. In some examples, thepattern generator 124 determines an initial scale estimate (e.g., adefault scale) for the new pattern. The scale estimate (e.g., in px/mm)is based on the pixel dimensions (width and height in pixels) of thepattern and the estimated dimensions of the object represented in thepattern (e.g., based on the estimate dimensions of the image from whichthe pattern was created). The dimensions of the object represented inthe pattern are typically unknown. In particular, while the truedimensions of a product may be known, the object represented in thepattern does not always correspond to the exact boundaries of theproduct. As such, the dimensions of the object represent in the patternare estimated. Therefore, an estimated scale may be assigned to thepattern. In some examples, the estimated scale is based on the truedimensions of the product (if known). In other examples, the estimatedscale may be based on an estimated dimension of the object as determinedfrom an estimated dimension from the original image from which thepattern was created. When the pattern is matched with objects in otherimages, more scale estimates can be assigned to the pattern to create ascale estimate array. The scale estimates tend to converge around thetrue scale of the pattern and can be used to more accurately determinethe true dimension of the object represented by the pattern.

In some examples, the auditor creates patterns from a shelf image afterthe shelf image has been processed by the image recognition application112. For example, after the image recognition process, one or moreproducts in the shelf image may not have been matched because there wasno corresponding pattern for the product in the pattern database 116. Assuch, the auditor can create a pattern for the corresponding product. Anexample method for creating a pattern is described in connection withFIG. 3.

FIG. 1 illustrates example patterns 114 a-n stored in the patterndatabase 116. The patterns 114 a-n includes an example first pattern 114a, which is an image of a bottle of Joe's Cider. The example firstpattern 114 a has a pixel dimension (width and height) of 200 px by 500px. In the illustrated example of FIG. 1, the first pattern 114 aincludes an array of stored scale estimates, including 2 px/mm, 2.1px/mm, and 2.3 px/mm. The example scale estimates are approximations onwhat scale the example first pattern 114 a represents. In otherexamples, the array of stored scale estimates may include more or fewerscale estimates.

Also illustrated in FIG. 1 is an example second pattern 114 b, which isa bottle of Bob's Beer. The example second pattern 114 b has a pixeldimension (width and height) of 175 px by 450 px. The second pattern 114b includes an array of stored scale estimates, including 3 px/mm, 3.5px/mm, and 3.8 px/mm. In other examples, the array of stored scaleestimates may include more or fewer scale estimates. An example thirdpattern 114 c is also illustrated in FIG. 1. The example third pattern114 c has a pixel dimension (width and height) of 150 px by 300 px. Inthe illustrated example of FIG. 1, the third pattern 114 c has only onescale estimate, 4 px/mm, which may be the default scale estimate.Likewise, the other ones of the patterns 114 a-n may include similarinformation, containing any number of scale estimates.

In some examples, when performing an image recognition process on ashelf image, a category or sub-group of the patterns 114 a-n may beselected (e.g., launched). In some examples, selecting a category orsub-group of the patterns 114 a-n, instead of all of the patterns 114a-n, reduces the processing time of the image recognition application112 and increases the efficiency of the image recognition process. Forexample, if the shelf image 104 is from a liquor store, then patternsrelating to baby diapers do not need to be launched because suchproducts are not expected to be on the shelf 106 in the liquor store.The example patterns 114 a-n can be categorized based on the type ofproduct (e.g., clothing, food, outdoor accessories, etc.). In theillustrated example of FIG. 1, the image processing system 100 includesa launch database 126 where the patterns 114 a-n to be analyzed arestored. In some examples, a user may select (e.g., via the userinterface 118) a desired category or sub-group of the patterns 114 a-nto be launched. In some examples, changes to the information relating tothe patterns 114 a-n in the launch database 126 are updated to thepattern database 116, and vice versa.

To increase the likelihood that the example patterns 114 a-n arecorrectly matched to objects appearing in the example shelf image 104,the patterns 114 a-n are rescaled or resized to the same scale estimateas the shelf image 104. An example source image analyzer 128 determinesthe pixel dimensions (width and height) of the shelf image 104 andestimates a shelf image scale estimate (e.g., an initial common scale)based on an estimated dimension o of the shelf 106 appearing in theshelf image 104. For example, the shelf image 104 may have a pixeldimension (width and height) of 2,250 px by 3,375 px, and an estimateddimension of 1,500 mm by 2,250 mm. The source image analyzer 128determines the shelf image scale estimate to be 1.5 px/mm: (2,250px/1,500 mm+3,375 px/2,250)/2=1.5 px/mm. In some examples, the estimateddimension of the shelf 106 represented by the shelf image 104 is inputby an auditor (e.g., via the user interface 118). In other examples, thesource image analyzer 128 may perform an estimation using differentapproaches, such as image type or current pattern selection. Forexample, using the image type technique, the example source imageanalyzer 128 uses heuristic models based on rules depending on the typeof shelf being analyzed. For example, shelves for diapers may have acommon dimension (e.g., based on an average) that can be used bydefault. In some examples, an auditor specifies the type of shelf beinganalyzed, and the example source image analyzer 128 calculates theestimated dimensions for the shelf. Using the current pattern selectiontechnique, the dimensions of the shelf 106 can be estimated based on thecategory or sub-group of patterns that are being launched. For example,if patterns relating to beer cans are being analyzed, the example sourceimage analyzer 128 determines how many beer cans may typically appear ona shelf (e.g., a maximum number of beer cans), and determines anestimated dimension of the shelf based on the number of beer cans.

In some examples, prior to commencing the image recognition process, anexample pattern scale optimizer 130 determines optimized scales for eachof the example patterns 114 a-n (being launched) that have a thresholdnumber of scale estimates. For example, assume the threshold is twoscale estimates. Thus, for any of the example patterns 114 a-n havingtwo or more scale estimates, the example pattern scale optimizer 130determines an optimized scale for the corresponding pattern. In otherexamples, a different threshold value may be implemented. In someexamples, the optimized scale is a median of the scale estimates for thecorresponding pattern. For example, the first pattern 114 a has threescale estimates. The pattern scale optimizer 130 determines a median ofthe array of scales estimates, which is 2.1 px/mm. Thus, 2.1 px/mm isdetermined to be an optimized scale corresponding to the first pattern114 a. For the example second pattern 114 b, the example pattern scaleoptimizer 130 similarly determines the median of the array of scaleestimates, which is 3.5 px/mm. Thus, 3.5 px/mm is determined to be anoptimized scale corresponding to the example second pattern 114 b. Insome examples, for patterns that do not satisfy the threshold, the scaleof the corresponding pattern is set to unknown (such that the pattern isnot used in determining a scale correction value, discussed in furtherdetail here). For example, with regards to the example third pattern 114c, there is only one scale estimate. Thus, the example third pattern 114c does not satisfy the threshold (e.g., having two or more scaleestimates) so the example pattern scale optimizer 130 sets the optimizedscale for the example third pattern 114 c to default (e.g., unknown) (todistinguish the third pattern 114 c from the other patterns 114 a-n thatmeet the threshold and have optimized scales). The example pattern scaleoptimizer 130 continues to determine optimized scales for all of thepatterns 114 a-n that meet the threshold. The optimized scales may bestored with the corresponding patterns 114 a-n in the example launchdatabase 126. In other examples, instead of using a median of the scaleestimates for the optimized scale, the pattern scale optimizer 130 mayuse another metric, such as an average, a cluster by applyingstatistical distances to minimize intra-cluster values (e.g., usingMahalanobis or Bhattacharyya distances) or another statistical model. Insome examples, after the pattern scale optimizer 130 has analyzed thepatterns 114 a-n, there is a first set of patterns (those with optimizedscale values (e.g., the first pattern 114 a)) and a second set ofpatterns (those with unknown (default) scales (e.g., the third pattern114 c)).

After the example shelf image scale estimate is determined and theoptimized scales for the example patterns 114 a-n are determined, theexample image processing system 100 resizes or rescales the examplepatterns 114 a-n to the shelf image scale estimate, e.g., 1.5 px/mm. Inthe illustrated example of FIG. 1, the image processing system 100includes a rescaler 132 to resize/rescale the patterns 114 a-n. Theexample rescaler 132 rescales/resizes both the patterns having optimizedscales (e.g., the first set of patterns (e.g., the first pattern 114 a,the second pattern 114 b, etc.)) and those patterns that have only onescale estimate (e.g., the second set of patterns that have scalesidentified as default (e.g., the third pattern 114 c)). As a result, theexample patterns 114 a-n are increased or decreased in size (pixeldimension). For example, if the optimized scale for the first pattern114 a is 2.1 px/mm (as determined by the pattern scale optimizer 130above), and the shelf image scale estimate is 1.5 px/mm (as determinedby the source image analyzer 128 above), the example rescaler 132transforms the first pattern 114 a based a scaling ratio or factor of0.71 (71%): 1.5 px/mm/2.1 px/mm=0.71. If the pixel dimension (width andheight) of the first pattern 114 a is 200 px by 500 px, then the pixeldimension (width and height) of the resized version of the first pattern114 a is about 142 px (200 px>0.71) by about 355 px (500 px×0.71). Insome examples, the example rescaler 132 resizes/rescales the firstpattern 114 a using bicubic interpolation to determine the new size ofthe new first pattern. In some examples, this technique ensures smoothresults where the pixels are created (e.g., during an upscale) from theneighboring pixels or mixed (e.g., during a downscale) into fewerpixels. In other examples, other techniques may be used. Each of theother ones of the patterns 114 a-n are similarly resized/rescaled (e.g.,based on a scaling ratio to increase or decrease the pixel dimensions)to match the shelf image scale estimate (e.g., 1.5 px/mm). In someexamples, the scaled versions of the patterns 114 a-n are stored in thelaunch database 126.

The example image recognition application 112 then analyzes the exampleshelf image 104 and detects or identifies objects in the shelf image 104that match the rescaled/resized versions of the patterns 114 a-n. FIG. 2illustrates an example segmented image 200 corresponding to the shelfimage 104 produced by the image recognition application 112. The examplesegmented image 200 may also be referred to as a segmentation. Theexample segmented image 200 includes one or more frames 202 a-n (e.g., aboundary, a rectangle, a grid, etc.) overlaid on top of the shelf image104. The example frames 202 a-n indicate the locations (e.g., positions,boundaries, regions, etc.) of objects 204 a-n (e.g., the products 108a-n) in the example shelf image 104 that have been matched with one ofthe patterns 114 a-n. In some examples, product information (e.g., aname of the respective product) may be displayed or presented along atop of the example frames 202 a-n. In some examples, the identifyinginformation included in respective ones of the example frames 202 a-n isassociated with the reference product(s) determined to match thecorresponding products 108 a-n depicted in the frames 202 a-n and may bestored in a database (e.g., the launch database 126) and/or included ina report (e.g., the report 122). In some examples, the match recordsinclude the example shelf image 104, metadata associated with thecaptured image (if present), and/or a product identifier as returnedfrom the reference record associated with the matching reference image(e.g., an MRE assigned product number, a UPC, etc.). The example matchrecords are associated with particular ones of the example frames 202a-n. In some examples, the information included in the match record(s)is used by an example report generator 120 to generate an example matchreport 122. In some examples, the match record(s) are stored in a matchdatabase and/or included in the match report 122 and/or are used by theexample image processing system 100 to generate shelf-audit reports.

In some examples, one or more of the objects 204 a-n matched with thepatterns 114 a-n may have a different size (e.g., pixel dimension inwidth and height) than the rescaled versions of the patterns 114 a-n.For example, as illustrated in FIG. 2, the first object 204 a wasidentified as match with the first pattern 114 a. However, the pixeldimensions of the example first object 204 a may not be the same pixeldimensions of the resized version of the first pattern 114 a. Forexample, assume the first object 204 a in FIG. 2 has a pixel dimension(width and height) of about 133 px×325 px. However, as mentioned above,the pixel dimensions of the rescaled version of the first pattern 114 aare about 142 px by 355 px. This deviation indicates an error in scaleestimate of the shelf image 104 (e.g., 1.5 px/mm) and/or the estimatescale (e.g., the optimized scale of 2.1 px/mm) of the first pattern 114a.

Referring back to FIG. 1, to improve the accuracy of the scale estimatesof the patterns 114 a-n and the accuracy of the image recognitionprocess, the image processing system 100 includes an example scalecorrector 134. The example scale corrector 134 analyzes the deviationvalues in size between the matched patterns 114 a-n (or the rescaledversions of the patterns 114 a-n) with optimized scales and thecorresponding objects in the shelf image 104, and calculates a scalecorrection value. In some examples, the scale corrector 134 performs astatistical analysis of all of the differences between the matchedpatterns and the corresponding objects in the shelf image 104. In someexamples, the scale corrector 134 uses the three-sigma rule of thumb(sometimes referred to as the 68-95-99.7 rule), which is a statisticalmeasure of the percentage of values that reside within a band around amean in a normal distribution with a width of one, two and threestandard deviations of the mean, which indicates a level of confidencein the obtained distribution. For example, assume the rescaled versionof the first pattern 114 a matched with the first object 204 a, which is0.06 (6%) larger than the rescaled version of the first pattern 114 a,the rescaled version of the second pattern 114 b matched with anotherobject, which is 0.03 (3%) smaller that the rescaled version of thesecond pattern 114 b, etc. The example scale corrector 134 identifiesthe differences in all of the scaled versions of the patterns 114 a-nfrom the matched objects and finds a common correction. For example, thedeviation value may result in an array such as [−0.06, −0.052, −0.02,−0.011, −0.008, +0.02, +0.02 . . . etc.]. For example, using thethree-sigma rule of thumb, assume the scale corrector 134 determinesthat the mean around which the scale changes are centered +0.01 (+1.0%).This scale correction value indicates that the shelf image scaleestimate is off by +1.0%. Therefore, the example scale corrector 134 mayadjust the shelf image scale estimate based on the scale correctionvalue. For example, if the original shelf image scale estimate was 1.5px/mm (as described in an example above), the adjusted or optimizedshelf image scale estimate is decreased by 0.01 (1.0%) to 1.485 px/mm.In other examples, the scale corrector 134 may determine that all of thedeviation values are centered around zero and, thus, no correction isneeded for the scale estimates. In some examples, significant outliers(e.g., outliers greater than 4-sigma) are ignored in the scalecorrection calculation.

Additionally, the example scale corrector 134 calculates a new scaleestimate for each of the example matched patterns 114 a-n havingoptimized scales (e.g., the first set of patterns) using the scalecorrection value. For instance, if the first object 204 a in FIG. 2 is133 px×325 px, and the adjusted shelf image scale estimate is 1.485px/mm, then the dimension of the item represented by the first object204 a is estimated to be about 90 mm wide (133 px/1.485 px/mm) by 220 mmhigh (325 px/1.485 px/mm). If the first pattern 114 a is 200 px by 500px, then the example scale corrector 134 determines a scale estimate forthe first pattern 114 a is about 2.2 px/mm: (200 px/90 mm+500 px/220mm)/2=˜2.2 px/mm. This new scale estimate for the first pattern 114 a isthen added to the array of scale estimates, such that the array of scaleestimates is now [2, 2.1, 2.2, 2.3]. Then, next time an imagerecognition process is performed, the optimized scale for the firstpattern 114 a may be 2.15 px/mm, which is the median of the array. Ascan be realized with this example approach, as more scale estimates areadded to the array, the optimized scales converge toward the true scaleand, thus, the true dimensions of the object represented in thecorresponding pattern.

Likewise, the example scale corrector 134 may calculate a new scaleestimate for each of the matched patterns 114 a-n that do not haveoptimized scales (e.g., the second set of patterns (e.g., the thirdpattern 103 c)). In this example, the new scale estimates are based onmatches between the patterns 114 a-n that had optimized scales (e.g.,more than two scale estimates), which increases the accuracy of thedeviation information and thus, increases the accuracy of the scalecorrection value. For example, assume the rescaled version of the thirdpattern 114 c is matched with a third object 204 c in the shelf image104 having a pixel dimension (width and height) of 60 px by 112 px.Using the adjusted shelf image scale estimate of 1.485 px/mm, theestimated dimensions of the third object 204 c are about 40 mm wide (60px/1.485 px/mm) by 75 mm high (112 px/1.485 px/mm). Therefore, if thepixel dimensions of the third pattern 114 c are 150 px by 300 px (FIG.1), a new scale estimate is about 3.875 px/mm: (150 px/40 mm+300 px/75mm)/2=˜3.875 px/mm. This new scale estimate can then be added to thescale estimate array for the third pattern 114 c. If the example thirdpattern 114 c is used again in another recognition, the third pattern114 c may meet the threshold value (e.g., two) and an optimized scalemay be calculated for the third pattern 114 c (because the third pattern114 c meets the threshold value). As such, the optimized scale for thethird pattern 114 c may be used in calculating the overall scalecorrection value as described above. Thus, the scale correction value isbased on scale estimates of those patterns having a certain number ofpreviously stored scaled estimates, which increases the reliability ofthe image recognition process. In some examples, after the imagerecognition process is performed a first time, the image recognitionprocess may be performed one or more subsequent times using the updatedscale information. For example, the image recognition produces may beperformed with the adjusted shelf image scale estimate (1.485 px/mm) andthe new scale estimates for the patterns 114 a-n.

As a pattern is used in more recognitions, the estimated scales for thepattern tend to converge toward the true scale of the pattern, therebycreating a more accurate scale estimate for the pattern. As a result,when the pattern is used for an image recognition process, the patterncan be rescaled/resized more accurately, which enables the imagerecognition process to be more accurate. In particular, the imagerecognition process is more accurate when a pattern can berescaled/resized to the same scale as the source image. Further, theexample image processing system 100 increases the accuracy of theestimation of the dimensions of the objects represented by the patternsand, thus, those matching objects in a shelf image. As a result, theexample image processing system 100 can better estimate the share of ashelf that a particular product or brand of product occupies. Thisinformation is important in determining compliance with particularquotas as outlined in contracts between the store and the manufacturers.For example, a manufacturer and a store may have a contract requires thestore to use 30% of a shelf for manufacturer's products. While theexample image recognition process is described in connection with ashelf image for shelf auditing, the example techniques disclosed hereincan likewise be applied to any other type of image recognition processwhere one or more patterns (e.g., reference images) are to be comparedto a source image to identify matches.

While an example manner of implementing the image processing system 100is illustrated in FIG. 1, one or more of the elements, processes and/ordevices illustrated in FIG. 1 may be combined, divided, re-arranged,omitted, eliminated and/or implemented in any other way. Further, theexample image recognition application 112, the example pattern database116, the example report generator 120, the example pattern generator124, the example launch database 126, the example source image analyzer128, the example pattern scale optimizer 130, the example rescaler 132,the example scale corrector 134 and/or, more generally, the exampleimage processing system 100 of FIG. 1 may be implemented by hardware,software, firmware and/or any combination of hardware, software and/orfirmware. Thus, for example, any of the example image recognitionapplication 112, the example pattern database 116, the example reportgenerator 120, the example pattern generator 124, the example launchdatabase 126, the example source image analyzer 128, the example patternscale optimizer 130, the example rescaler 132, the example scalecorrector 134 and/or, more generally, the example image processingsystem 100 could be implemented by one or more analog or digitalcircuit(s), logic circuits, programmable processor(s), applicationspecific integrated circuit(s) (ASIC(s)), programmable logic device(s)(PLD(s)) and/or field programmable logic device(s) (FPLD(s)). Whenreading any of the apparatus or system claims of this patent to cover apurely software and/or firmware implementation, at least one of theexample image recognition application 112, the example pattern database116, the example report generator 120, the example pattern generator124, the example launch database 126, the example source image analyzer128, the example pattern scale optimizer 130, the example rescaler 132,and/or the example scale corrector 134 is/are hereby expressly definedto include a tangible computer readable storage device or storage disksuch as a memory, a digital versatile disk (DVD), a compact disk (CD), aBlu-ray disk, etc. storing the software and/or firmware. Further still,the example image processing system 100 of FIG. 1 may include one ormore elements, processes and/or devices in addition to, or instead of,those illustrated in FIG. 1, and/or may include more than one of any orall of the illustrated elements, processes and devices.

Flowcharts representative of example machine readable instructions forimplementing the image recognition application 112, the example patterndatabase 116, the example report generator 120, the example patterngenerator 124, the example launch database 126, the example source imageanalyzer 128, the example pattern scale optimizer 130, the examplerescaler 132, the example scale corrector 134 and/or, more generally,the example image processing system 100 are shown in FIGS. 3, 4A and 4B.In these examples, the machine readable instructions comprise a programfor execution by a processor such as the processor 512 shown in theexample processor platform 500 discussed below in connection with FIG.5. The programs may be embodied in software stored on a tangiblecomputer readable storage medium such as a CD-ROM, a floppy disk, a harddrive, a digital versatile disk (DVD), a Blu-ray disk, or a memoryassociated with the processor 512, but the entire programs and/or partsthereof could alternatively be executed by a device other than theprocessor 512 and/or embodied in firmware or dedicated hardware.Further, although the example programs are described with reference tothe flowcharts illustrated in FIGS. 3, 4A and 4B, many other methods ofimplementing the example image recognition application 112, the examplepattern database 116, the example report generator 120, the examplepattern generator 124, the example launch database 126, the examplesource image analyzer 128, the example pattern scale optimizer 130, theexample rescaler 132, the example scale corrector 134 and/or, moregenerally, the example image processing system 100 may alternatively beused. For example, the order of execution of the blocks may be changed,and/or some of the blocks described may be changed, eliminated, orcombined.

As mentioned above, the example processes of FIGS. 3, 4A and 4B may beimplemented using coded instructions (e.g., computer and/or machinereadable instructions) stored on a tangible computer readable storagemedium such as a hard disk drive, a flash memory, a read-only memory(ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, arandom-access memory (RAM) and/or any other storage device or storagedisk in which information is stored for any duration (e.g., for extendedtime periods, permanently, for brief instances, for temporarilybuffering, and/or for caching of the information). As used herein, theterm tangible computer readable storage medium is expressly defined toinclude any type of computer readable storage device and/or storage diskand to exclude propagating signals and to exclude transmission media. Asused herein, “tangible computer readable storage medium” and “tangiblemachine readable storage medium” are used interchangeably. Additionallyor alternatively, the example processes of FIGS. 3, 4A and 4B may beimplemented using coded instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media. As usedherein, when the phrase “at least” is used as the transition term in apreamble of a claim, it is open-ended in the same manner as the term“comprising” is open ended.

FIG. 3 is a flowchart representative of an example method 300 forcreating patterns and which may be implemented the example imageprocessing system 100 of FIG. 1. In some examples, the example method300 corresponds to a workflow for creating new patterns that may beperformed by an auditor. At block 302, the example image processingsystem 100 retrieves a new shelf image such as, for example, the shelfimage 104. In some examples, the shelf image 104 is provided by theimage capturing device 102. An auditor may view the example shelf image104 on the user interface 118. In some examples, the shelf image 104 isan image that has been previously analyzed by the image recognitionapplication 112. For example, as mentioned above, in some instances theimage recognition application 112 does not recognize all of the products108 a-n in the shelf image 104. When reviewing the segmented image 200,an auditor may see that one of the products in the segmented image 200has not been recognized. The auditor may highlight or otherwise select aregion of interest (ROI) (e.g., an object) in the image corresponding toa product or portion of a product (e.g., a label) for which the auditorwants to create a pattern.

At block 304, the example pattern generator 124 detects a ROI identified(e.g., selected, highlighted, etc.) by the auditor. At block 306, theexample image processing system 100 saves the portion of the imagecorresponding to the ROI as a pattern in the example pattern database116. The auditor may identify a product, branch, etc. associated withthe pattern. In some examples, the auditor selects the information forma menu or list of options. At block 308, the example image processingsystem 100 saves product information (e.g., name of the product, type ofpackaging, number of products in each package, etc.) related to thenewly created pattern. In some examples, an auditor manually enters theproduct information

At block 310, the example pattern generator 124 determines (e.g., sets)metric information for the pattern. For example, the pattern generatormay calculate a scale estimate for the pattern based on the pixeldimensions (width and height) of the pattern and the estimateddimensions of the object represented in the pattern. In some examples,the estimated dimensions of the object are based on estimated dimensionsof the source image (e.g., the shelf image 104 from which the patternwas created). In other examples, the true dimensions of the product maybe known and used to calculate the initial scale estimate. However, asdescribed above, using the true dimensions of the product may not resultin an accurate scale as the boundaries of the pattern are not alwaysaccurate. At block 312, the example image processing system 100determines whether another pattern is to be created. If so, the examplemethod 300 can be repeated. Otherwise, the example method 300 ends atblock 314.

FIGS. 4A and 4B are flowcharts representative of example methods 400 foranalyzing a shelf image (e.g., performing a shelf audit) and which maybe implemented by the example image processing system 100 of FIG. 1. Atblock 402 in FIG. 4A, the example image processing system 100 retrievesthe shelf image 104 (e.g., from the image capturing device 102). Theshelf image 104 is to be analyzed by an image recognition application toidentify products and other associated information in the shelf image104 as part of an audit. At block 404, the example image processingsystem 100 loads the patterns to be launched. For example, as mentionedabove, the patterns 114 a-n in the pattern database 116 may includepatterns not relevant to a particular audit. Therefore, a category orsub-group of the patterns 114 a-n to be analyzed may be stored in launchdatabase 126.

At block 406 of the example method 400, the pattern scale optimizer 130determines whether a pattern being launched satisfies (e.g., meets,exceeds, etc.) a threshold number of scale estimates. In some examples,the threshold is two scale estimates. Thus, for example, the firstpattern 114 a, which has three scale estimates, satisfies the threshold.In other examples, other thresholds may be used (e.g., 5, 8, etc.). If apattern satisfies the threshold, the example pattern scale optimizer 130determines an optimized scale for the corresponding pattern at block408. In some examples, the optimized scale is a median of the scaleestimate array for the corresponding pattern. For example, the median ofthe scale estimate array of the first pattern 114 a is 2.1 px/mm. Inother examples, another metric (e.g., an average) may be used tocalculate an optimized scale. If a pattern does not satisfy thethreshold (e.g., the pattern only has one scale estimate), the examplepattern scale optimizer 130 sets the scale of the pattern to unknown(e.g., default) at block 410. At block 412, the example pattern scaleoptimizer 130 saves the scale information (the optimized scale or thedefault scale) of the pattern (e.g., in the launch database 126). Atblock 414, the example pattern scale optimizer 130 determines if thereis another pattern to be analyzed. If so, the example method 400proceeds to block 406 for the next pattern. If all of the patterns to belaunched have been analyzed, the example method 400 proceeds to block416. In some examples, after the pattern scale optimizer 130 hasanalyzed the patterns, there is a first set of patterns (those withoptimized scales) and a second set of patterns (those with unknown(default) scales).

At block 416, the example image processing system 100 launches the imagerecognition task, e.g., by activating the example image recognitionapplication 112. At block 418, the example source image analyzer 128determines the shelf image scale estimate for the example shelf image104 based on the pixel dimensions (width and height) of the shelf image104 and an estimated dimension of the shelf 106 represented in the shelfimage 104. For example, as mentioned above in connection with FIG. 1,the shelf image 104 may have a scale estimate of 1.5 px/mm.

At block 420, the example rescaler 132 rescales or resizes the examplepatterns 114 a-n to the same scale as the shelf image scale estimate.For example, if the optimized scale for the first pattern 114 a is 2.1px/mm (as determined by the pattern scale optimizer 130 at block 408),and the shelf image scale estimate is 1.5 px/mm (as determined by thesource image analyzer 128 at block 418), the rescaler 132 transforms thefirst pattern 114 a based on a scaling ratio or factor of 0.71 (71%).The example rescaler 132 rescales or resizes all of the patterns withoptimized scales (based on the corresponding optimized scales) and thepatterns without optimized scales (based on the single scale estimatefor the corresponding pattern). After block 420, the example method 400continues to FIG. 4B.

At block 422 in FIG. 4B, the example image recognition application 112analyzes the example shelf image 104 and identifies the objects 204 a-nin the shelf image 104 that match the example patterns 114 a-n (e.g.,the resized/rescaled versions of the patterns 114 a-n). In someexamples, the image recognition application 112 creates a segmentedimage, such as the segmented image 200 of FIG. 2, which displays frames202 a-n around the objects 204 a-n matched with the patterns 114 a-n.

As mentioned above, in some examples, the objects 204 a-n in the shelfimage 104 that are matched to the patterns 114 a-n are a different size(in pixels) than the rescaled/resized versions of the patterns 114 a-n.At block 424, the scale corrector 134 calculates the deviation values(e.g., the differences) between the sizes of the objects 204 a-n in theshelf image 104 (e.g., a first set of the objects 204 a-n) and therescaled/resized versions of the patterns 114 a-n for those patternshaving an optimized scaled (e.g., those patterns with two or moreprevious scale estimates, a first set of the patterns 114 a-n)). Inother words, for each scaled version of a pattern having an optimizedscale and that has been matched to an object in the shelf image 104, theexample scale corrector 134 determines a difference in size (if any)between the scaled version of the pattern and the matched object in theshelf image 104. Therefore, in some examples, the scale corrector 134does not calculate the difference in size between the objects in theshelf image 104 that matched the rescaled versions of the patternswithout optimized values (e.g., those patterns that did not satisfy thethreshold at block 406). As such, only those patterns having optimizedscales, which are more reliable, are used to determine the scalecorrection value.

At block 426, the example scale corrector 134 calculates a scalecorrection value for the shelf image scale estimate based on thedeviation values (as determined at block 424). In some examples, thescale correction value is based on a three-sigma rule. If the deviationvalues fall around zero, for example, then no correction may be needed.However, if the deviation values fall around a particular (e.g.,threshold) positive or negative value, then a scale correction value isdetermined. For instance, if the average deviation value is +0.01 (+1%),then the scale correction value is −0.01 (−1%).

At block 428, the example scale corrector 134 determines an adjusted orcorrected scale estimate for the example shelf image 104 based on thescale correction value (as determined at block 426). For example, if theoriginal scale estimate of the shelf image 104 was 1.5 px/mm, the scalecorrector 134 decreases the original estimate by the scale correctionvalue 0.01 (−1%) to result an adjusted scale estimate of 1.485 px/mm.

At block 430, the example scale corrector 134 determines a new scaleestimate for each of the example patterns 114 a-n matched in the exampleshelf image 104 based on the size of the corresponding object and usingthe scale correction value and/or the adjusted scale estimate of theshelf image 104. For example, as illustrated in FIG. 2, the firstpattern 114 a was matched with the first object 204 a in the shelf image104. The example scale corrector 134 can determine an estimateddimension of the example first object 204 a based on the pixel dimensionof the first object 204 a and the adjust shelf image scale estimate. Theestimated dimensions can then be used to calculate a new scale estimatefor the first pattern 114 a based on the pixel dimensions (e.g., 200 pxby 500 px) of the first pattern 114 a.

At block 432, the example image processing system 100 saves the newscale estimates (as determined at block 430) for the respective patterns114 a-n in the pattern database 116. As can be realized, as morerecognitions occur and more scale estimates are added to the scaleestimate arrays, the average of the arrays converge toward a truedimension of the respective pattern. Thus, when the pattern is usedagain in another recognition, the optimized scale value is closer to theactual value and, thus, can be more accurately resized/rescaled to thedimensions of the image. This enhances the ability of the imagerecognition application 112 to match objects in the shelf image 104 andreduces errors (e.g., reduces false positives) when identifying suchobjects.

At block 434, the example image processing system 100 determines whetherto reanalyze the example shelf image 104. If so, the method 400continues to block 404 in FIG. 4A. During the second analysis, some ofthe example patterns 114 a-n that previously did not satisfy thethreshold at block 406 (e.g., the third pattern 114 c), may now meet thethreshold. Those patterns can then be more correctly resized/rescaled(e.g., at block 420). As a result, there is a greater chance thatobjects in the shelf image 104 are matched with the correspondingpattern. The example method 400 can be repeated numerous times. Eachiteration refines the results and leads to a more accurate imagerecognition process. In some examples, after the image recognitionprocess is complete and the values are updated, the example reportgenerator 120 generates the example report 122 and/or the results aredisplayed on the user interface 118. Otherwise, if the shelf image 104is not re-analyzed, the method 400 ends at block 436.

FIG. 5 is a block diagram of an example processor platform 500 capableof executing the instructions of FIGS. 3, 4A and 4B to implement theexample image processing system 100 of FIG. 100. The processor platform500 can be, for example, a server, a personal computer, a mobile device(e.g., a cell phone, a smart phone, a tablet such as an iPad™), apersonal digital assistant (PDA), an Internet appliance, a gamingconsole, a personal video recorder, or any other type of computingdevice.

The processor platform 500 of the illustrated example includes aprocessor 512. The processor 512 of the illustrated example includeshardware that may implement one or more of the example image recognitionapplication 112, the example pattern database 116, the example reportgenerator 120, the example pattern generator 124, the example launchdatabase 126, the example source image analyzer 128, the example patternscale optimizer 130, the example rescaler 132, and/or the example scalecorrector 134 of example image processing system 100 of FIG. 1. Forexample, the processor 512 can be implemented by one or more integratedcircuits, logic circuits, microprocessors or controllers from anydesired family or manufacturer.

The processor 512 of the illustrated example includes a local memory 513(e.g., a cache). The processor 512 of the illustrated example is incommunication with a main memory including a volatile memory 514 and anon-volatile memory 516 via a bus 518. The volatile memory 514 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM)and/or any other type of random access memory device. The non-volatilememory 516 may be implemented by flash memory and/or any other desiredtype of memory device. Access to the main memory 514, 516 is controlledby a memory controller.

The processor platform 500 of the illustrated example also includes aninterface circuit 520. The interface circuit 520 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), and/or a PCI express interface.

In the illustrated example, one or more input devices 522 are connectedto the interface circuit 520. The input device(s) 522 permit(s) a userto enter data and commands into the processor 512. The input device(s)can be implemented by, for example, an audio sensor, a microphone, acamera (still or video), a keyboard, a button, a mouse, a touchscreen, atrack-pad, a trackball, isopoint and/or a voice recognition system.

One or more output devices 524 are also connected to the interfacecircuit 520 of the illustrated example. The output devices 524 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay, a cathode ray tube display (CRT), a touchscreen, a tactileoutput device, a printer and/or speakers). The interface circuit 520 ofthe illustrated example, thus, typically includes a graphics drivercard, a graphics driver chip or a graphics driver processor.

The interface circuit 520 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem and/or network interface card to facilitate exchange of data withexternal machines (e.g., computing devices of any kind) via a network526 (e.g., an Ethernet connection, a digital subscriber line (DSL), atelephone line, coaxial cable, a cellular telephone system, etc.).

The processor platform 500 of the illustrated example also includes oneor more mass storage devices 528 for storing software and/or data.Examples of such mass storage devices 528 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, RAIDsystems, and digital versatile disk (DVD) drives.

Coded instructions 532 to implement the methods of FIGS. 3, 4A and 4Bmay be stored in the mass storage device 528, in the volatile memory514, in the non-volatile memory 516, and/or on a removable tangiblecomputer readable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that the above disclosedmethods, apparatus/systems and articles of manufacture achieve betterrecognition rates than known image recognition technologies. Examplesdisclosed herein reduce (e.g., minimize) the differences in scalesbetween patterns and the image (and/or objects in the image) to whichthe patterns are being compared. Examples disclosed herein also utilizeoptimized scales for the patterns, which are more reliable than thedefault scales typically assigned to a pattern. As a result, the initialbasis for the scaling ratio is more accurate and leads to a betterrecognition process. Further, example systems and methods can be used todetermine a more accurate estimate of the dimension(s) of a product,which is desirable for performing image recognition and for performingan in-store audit.

Although certain example methods, apparatus/systems and articles ofmanufacture have been disclosed herein, the scope of coverage of thispatent is not limited thereto. On the contrary, this patent covers allmethods, apparatus/systems and articles of manufacture fairly fallingwithin the scope of the claims of this patent.

What is claimed is:
 1. An apparatus to reduce scale estimate errors, theapparatus comprising: an image recognition application to identifymatches between stored patterns and objects detected in a shelf image,the shelf image having a shelf image scale estimate; and a scalecorrector to: calculate deviation values between sizes of (A) a firstset of the objects detected in the shelf image and (B) a first set ofthe stored patterns matched with the first set of the objects; andreduce an error of the shelf image scale estimate by calculating a scalecorrection value for the shelf image scale estimate based on thedeviation values.
 2. The apparatus as defined in claim 1, wherein thescale corrector is to calculate the deviation values by determining adifference in pixel dimensions of a first scaled version of a firstpattern of the first set of the stored patterns and a first object ofthe first set of the objects matched with the first pattern.
 3. Theapparatus as defined in claim 2, further including a rescaler togenerate the first scaled version of the first pattern by rescaling thefirst pattern based on the shelf image scale estimate.
 4. The apparatusas defined in claim 1, wherein the first set of the stored patterns haveoptimized scales.
 5. The apparatus as defined in claim 4, furtherincluding a pattern scale optimizer to determine the optimized scalesfor the first set of the stored patterns.
 6. The apparatus as defined inclaim 4, wherein the optimized scales are a median of estimated scalesfor the first set of the stored patterns.
 7. The apparatus as defined inclaim 1, wherein the stored patterns include a second set of the storedpatterns, the first set of the stored patterns having a number of scaleestimates that satisfy a threshold, the second set of the storedpatterns having a number of scale estimates that do not satisfy thethreshold.
 8. The apparatus as defined in claim 1, wherein the storedpatterns are image representations of at least a portion of a product orpackaging of a product, respective ones of the first set of the storedpatterns having stored scale estimates associated with the respectivestored pattern.
 9. The apparatus as defined in claim 8, wherein each ofthe first set of the stored patterns has a threshold number of storedscale estimates associated with the respective stored pattern.
 10. Theapparatus as defined in claim 1, wherein the scale corrector is tocalculate additional scale estimates for respective ones of the firstset of the stored patterns matched with the first set of the objectsbased on the sizes of the first set of the objects detected in the shelfimage and the scale correction value for the shelf image scale estimate.11. A method to reduce scale estimate errors, the method comprising:identifying, by executing an instruction with a processor, matchesbetween stored patterns and objects detected in a shelf image, the shelfimage having an shelf image scale estimate; calculating, by executing aninstruction with the processor, deviation values between sizes of (A) afirst set of the objects detected in the shelf image and (B) a first setof the stored patterns matched with the first set of the objects; andreducing an error of the shelf image scale estimate by calculating, byexecuting an instruction with the processor, a scale correction valuefor the shelf image scale estimate based on the deviation values. 12.The method as defined in claim 11, wherein calculating the deviationvalues includes determining a difference in pixel dimensions of a firstscaled version of a first pattern of the first set of the storedpatterns and a first object of the first set of the objects matched withthe first pattern.
 13. The method as defined in claim 12, furtherincluding generating the first scaled version of the first pattern byrescaling the first pattern based on the shelf image scale estimate. 14.The method as defined in claim 11, wherein the first set of the storedpatterns have optimized scales.
 15. The method as defined in claim 14,further including determining the optimized scales for the first set ofthe stored patterns.
 16. The method as defined in claim 14, wherein theoptimized scales are a median of estimated scales for the first set ofthe stored patterns.
 17. The method as defined in claim 11, wherein thestored patterns include a second set of the stored patterns, the firstset of the stored patterns having a number of scale estimates thatsatisfy a threshold, the second set of the stored patterns having anumber of scale estimates that do not satisfy the threshold.
 18. Atangible machine readable storage medium comprising instructions that,when executed, cause a machine to at least: identify matches betweenstored patterns and objects detected in a shelf image, the shelf imagehaving an shelf image scale estimate; calculate deviation values betweensizes of (A) a first set of the objects detected in the shelf image and(B) a first set of the stored patterns matched with the first set of theobjects; and reduce an error of the shelf image scale estimate bycalculating a scale correction value for the shelf image scale estimatebased on the deviation values.
 19. The tangible machine readable storagemedium as defined in claim 18, wherein the instructions, when executed,cause the machine to calculate the deviation values by determining adifference in pixel dimensions of a first scaled version of a firstpattern of the first set of the stored patterns and a first object ofthe first set of the objects matched with the first pattern.
 20. Thetangible machine readable storage medium as defined in claim 19, whereinthe instructions, when executed, further cause the machine to generatethe first scaled version of the first pattern by rescaling the firstpattern based on the shelf image scale estimate.
 21. The tangiblemachine readable storage medium as defined in claim 18, wherein thefirst set of the stored patterns have optimized scales.
 22. The tangiblemachine readable storage medium as defined in claim 21, wherein theinstructions, when executed, further cause the machine to determine theoptimized scales for the first set of the stored patterns.
 23. Thetangible machine readable storage medium as defined in claim 18, whereinthe stored patterns include a second set of the stored patterns, thefirst set of the stored patterns having a number of scale estimates thatsatisfy a threshold, the second set of the stored patterns having anumber of scale estimates that do not satisfy the threshold.